Feature or Bugfix

• Bugfix

Detail

• athena2pyarrow rejected Athena varbinary and incorrectly accepted substrings of "binary" (for example "i") because the membership check used ("binary" or "varbinary"), which evaluates to the string "binary".
• Align the check with athena2pandas so both binary and varbinary map to pa.binary(), and invalid substrings raise UnsupportedType.
• Add unit regression coverage in tests/unit/test_data_types.py.

Relates

• Closes athena2pyarrow rejects Athena varbinary and accepts invalid substrings #3412

Summary

athena2pyarrow had a long-standing operator typo in the binary-type branch. This prevented casting Athena varbinary columns (for example via dtype={"col": "varbinary"} on Parquet writes) and made substring values such as "i"/"n"/"ary" incorrectly resolve to pa.binary().

Reproduction

from awswrangler._data_types import athena2pyarrow

athena2pyarrow("varbinary")  # UnsupportedType before fix
athena2pyarrow("i")          # pa.binary() before fix (should raise)

No AWS credentials are required.

Root cause

if dtype in ("binary" or "varbinary"):  # -> dtype in "binary"

("binary" or "varbinary") is the string "binary", so membership becomes substring membership.

Implementation

One-line change in awswrangler/_data_types.py:

if dtype in ("binary", "varbinary"):

matching the existing athena2pandas contract.

Regression tests

tests/unit/test_data_types.py:

• binary / varbinary (case-insensitive) → pa.binary()
• substrings "i", "n", "ary", "bin" → UnsupportedType
• parity check that athena2pandas still accepts both binary types

Validation

• uv run pytest tests/unit/test_data_types.py -v → 10 passed
  • Confirmed 6 failures on clean upstream/main before the fix
• uv run pytest tests/unit/test_metadata.py tests/unit/test_session.py tests/unit/test_utils.py tests/unit/test_data_types.py -q → 30 passed
• AWS_DEFAULT_REGION=us-east-1 uv run pytest -n 4 tests/unit/test_moto.py -q → 45 passed
• uv run ruff format --check . → pass
• uv run ruff check . → pass
• uv run mypy --install-types --non-interactive awswrangler/_data_types.py → pass
• uv run doc8 --ignore-path docs/source/stubs --max-line-length 120 docs/source → pass
• uv lock --check → pass
• uv build → built sdist and wheel
• git diff --check → pass

AWS integration / credential-dependent tests not run

• Full tests/unit/test_s3_parquet.py and other live-AWS suites require deployed test infrastructure / credentials (as documented in CONTRIBUTING.md). Errors observed without that environment are botocore credential/region failures and are unrelated to this change.

Pre-existing local findings (clean upstream/main)

• uv run mypy awswrangler reports 20 existing errors in unrelated Ray/OpenSearch/etc. modules on this local toolchain; unchanged by this PR. CI Static Checking remains the authoritative gate.

Compatibility

• Public APIs unchanged.
• Callers that already passed binary continue to work.
• Callers that pass Athena varbinary now succeed instead of raising.
• Invalid substring inputs that previously returned pa.binary() now raise UnsupportedType (correctness fix).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.